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Cartoon Recognition 

This Invention relates to a method of and apparatus for determining whether an Image, for 
example a frame of a video signal, represents a cartoon. 

5 

With the growing availability of online data, provision of hundreds or even thousands of 
data channels by an Infomiation provider causes problems of content management and 
verification, as manual checking of every piece of data becomes infeaslble. For Image 
data, there is increasing interest in techniques for automated image interpretation and 
1 0 classification. Automated image interpretation and classification could help with Indexing, 
cataloging and searching of still image or moving image databases. 

Image interpretation and classification can be done either by the service provider or by the 
service receiver. For example. If It is possible to detemiine whether a signal represents a 
1 5 cartoon or not then it is possible for parents to stop children from downloading pictures 
from the Internet or from watching TV programs other than cartoons. Other types of 
classifiers could prove useful, for example, classification of pornographic images or 
recognition of particular people. 

20 According to the present invention there is provided a method for classifying whether an 
image represents a cartoon, comprising the step of generating a likelihood in dependence 
on the presence of low luminosity outlines in the image. 

According to the present Invention there is also provided a data carrier loadable into a 
25 computer and carrying instrucOons for causing the computer to carry out said method. 

in a preferred embodiment the method further comprises the step of analysing the Image 
to provide one or more parameters wherein one parameter relates to the luminosity of the 
Image; and the generating step detennines the generated likelihood in dependence upon 
30 the value of said one parameter. 

Advantageously the image comprises a plurality of pixels and the analysing step includes 
the sub-step of vector quantising the Image so that each pixel corresponds to one of a 
plurality of codes. 

35 



Preferably the analysing step further comprises the sub-step of calculating the percentage 
of pixels corresponding to a one of the plurality of codes and preferably said one of the 
plurality of codes is a code which conresponds to pixels of low luminosity. 

5 In a prefenred embodiment the vector quantising sub-step comprises sub-steps of dividing 
the image into a plurality of blocks, each block comprising a subset of pixels in the image; 
and independently vector quantising each block. 

Advantageously the generating step comprises the sub-step of generating a block 
10 likelihood value for each of a plurality of blocks, the block likelihood representing the 
probability that the pixels in that block represent an image comprising one or more 
outlines and preferably the generating step comprises the sub-step of combining a 
plurality of block likelihood values to provide the likelihood value for the image. 

1 5 According to another aspect of the present invention there is provided apparatus for 
classifying whether an image represents a cartoon said apparatus comprising generating 
means for generating a likelihood in dependence on the presence of low luminosity 
outlines in the image. 

20 According to the present invention there is also provided a data carrier loadable into a 
computer and carrying instructions for enabling the computer to provide said apparatus. 

In a preferred embodiment the apparatus further comprises means for analysing the 
image to provide one or more parameters wherein one parameter relates to the luminosity 
25 of the image; and the generating means receives in operation said one parameter and 
determines the generated likelihood in dependence upon the value of said one parameter. 

Advantageously the analysing means comprises a vector quantiser which receives in 
operation a plurality of pixels comprising the image and outputs a plurality of codes each 
30 output code corresponding to each of the received pixels. 

Preferably the analysing means further comprises means for calculating the percentage of 
pixels con^esponding to a one of the plurality of codes and said one of the plurality of 
codes is a code which corresponds to pixels of low luminosity. 
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In a preferred embodiment the vector quantiser further comprises means for dividing the 
image into a plurality of blocks, each block comprising a subset of pixels in the image; and 
means for independently vector quantising each block. 

5 Advantageously the generating means comprises means for generating a block likelihood 
value for each of a plurality of blocks, the block likelihood value representing the 
probability that the pixels in that block represent an image comprising one or more 
outlines and the generating means further comprises means for combining a plurality of 
block likelihood values to provide a likelihood value for jthe image. 



An embodiment of the invention will now be described, by way of example only, with 
reference to the accompanying drawings in which 

Figure 1 is a schematic representation of a computer loaded with software embodying the 
1 5 present invention; 

Figure 2 shows red. blue, green and luminance components for a cartoon; 

Figure 3 shows red, blue, green and luminance components for a photograph; 

Figure 4 shows red, blue, green and luminance components for a complex cartoon; 

Figure 5 is a functional block diagram of the program elements that comprise the software 
20 indicated in Figure 1; 

Figure 6 is a flow chart showing the method steps performed in one embodiment of the 

invention by the software illustrated in Figure 5; 

Figure 7 is a flow chart showing the vector quantising step of the method illustrated in 
Figure 6; 

25 Figure 8 is a flow chart showing the production of a tow luminosity signal; 

Figure 9 shows images for each level of vector quantisation for a cartoon and a 
photograph; and 

Figure 10 is a flow chart showing the determination step of the method Illustrated in Figure 
6. 



Figure 1 illustrates a conventional computer 101. such as a Personal Computer, generally 
refen-ed to as a PC, running a conventional operating system 103, such as >/Vindows (a 
Registered Trade Marie of Microsoft Corporation), and having a number of resident 
application programs 105 such as a word processing program, a networic browser and e- 
35 mail program or a database management program. The computer 101 also includes an 
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image classification program 109 that enables a signal representing an image to be 
classified according to whether the image represents a cartoon. The computer 101 Is also 
connected to a conventional disc storage unit 111 for storing data and programs, a 
keyboard 113 and mouse 115 for allowing user input and a printer 117 and display unit 
119 for providing output from the computer 101. The computer 101 also has access to 
external networks (not shown) via a network card 121. 

A simple image classification program for determining whether an image represents a 
cartoon can be implemented which analyses the colour distribution within the image. 
Figure 2a shows a cartoon (represented in a grey scale in the figure). Figure 2b is a 
histogram showing the number of pixels with particular values for the red component 
Figures 2c, 2d and 2e show similar histograms for the green, blue and luminance 
components. For this cartoon there are prominent spikes in each component. Figure 3 
shows a similar set of histograms for an image which is not a cartoon. The distribution for 
each component shows no such spikes. However, for a more complicated cartoon, such 
as that shown in Figure 4, it is difficult to see any obvious distinction between the 
histograms of Figures 4b - 4e and those of Figures 3b - 3e. 

As shown in Figure 6, in accordance with a method of the present invention at step 10 an 
input signal representing an image, for example a frame of video data, comprising a 
plurality of pixels is received. At step 20 the received signal is converted into a luminosity 
signal, which represents a grey scale version of the image, by calculating a luminosity 
value (L) for each pixel. The received signal has components representing a value in the 
range 0 to 255 for a red component (R) a blue component (B) and a green component (G) 
for each of the plurality of pixels which comprise the frame of video data. The luminosity 
value is calculated at step 20 using the equation 

L = 0.299R + 0.587G + 0.1 14B 
It is not necessary to use a calculated luminosity value; the invention works equally well if 
a vector comprising, for example, the R, G and B values is used. At step 30 the image 
represented by the luminosity signal is split into a plurality of block signals, each signal 
representing an area of the original image. In the embodiment of the invention described 
here, each block signal represents an area of the same size as the area represented by 
each other block signal (although the size may differ slightly due to quantisation effects) 
However, the areas represented by the block signals could equally well be different sizes 
from each other. 



At step 40 each block signal is vector quantised into a predetemnined number of levels. A 
code (for example an integer in the range 1 to the predetermined number of levels) being 
used to represent each level. At step 50 the vector quantised signals are used to provide 
5 a low luminosity signal comprising the vector quantised signals which represent the 
darkest level for each block. Finally at step 60 the lowest luminosity signal is used to 
determine whether the received signal represents a cartoon. It is not necessary to split the 
luminosity signal into a plurality of block signals prior to vector quantisation. However, the 
determination at step 60 is more accurate if the received signal is split Into signals 
1 0 representing smaller blocks of the frame. 

The algorithm now described with reference to Rgure 7 is used to determine the level 
represented by each code and to assign each pixel value to a code. It is similar to the well 
known LBG algorithm (as described in Linde, Y. Buzo. A and Gray, R. M. "An algorithm for 

1 5 vector quantizer design", IEEE Trans. Comm., vol. COM-28, Jan 1980, S 84-86). In Figure 
7 at step 42 each pixel value is assigned to a code. Initially there is a single code used to 
represent each pixel value. At step 43, for the or each code, the mean and the standard 
deviation of the pixel values which the or each code currently represents are calculated. 
The mean for the or each code is then associated with that code. At step 44, the code 

20 which represents pixel values having the greatest standard deviation is determined. A new 
value to be associated with that code is then calculated at step 45 as the mean for that 
code minus half the standard deviation for that code. If the new value is calculated to be 
less than zero then the new value is set to zero. At step 46 a new value to be associated 
with a new code is calculated as the mean plus half said standard deviation. 

25 

At step 47 one of the plurality of codes is assigned to each pixel value. A code is used to 
represent a pixel value if the value which that code is associated with is 'closer* to the 
pixel value than any of the other codes. In this embodiment of the invention, a luminosity 
value is used, so it is a simple matter to measure the distance between the luminosity 
30 value and the value associated with a code, by calculating the difference between the two 
values. In embodiments which use a vector of values to represent each pixel, such as R, 
G, and B values, a distance may be calculated using, for example, the 'city-block' distance 
or the least squares distance. 

35 At step 48 a check is performed to check whether the number of codes corresponding to 
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the predetermined number of levels have been created. If not, the steps 43 to 47 are 
repeated. In the embodiment of the invention described here four codes are created for 
each block, although the number of codes (and hence the predetermined number of 
levels) does not need to be the same for each block. In embodiments of the invention 
5 using a vector of values for each pixel the vector quantising step operates In an analogous 
manner to that described above. 

In Figure 8 at step 51 each block is taken in turn. At step 52 the luminance value for each 
pixel is set to be equal to the value associated with the code which is used to represent 

1 0 that pixel. At step 53 a signal is generated with the luminance value for each pixel set to 
white for each pixel which is not represented by the code associated with the lowest 
luminosity value for that block. In the embodiment of the invention described here a similar 
signal is generated (for display purposes) for each one of the codes, in order to generate 
images for each vector quantisation level. The image generated from the signals for the 

1 5 lowest luminosity value will be referred to as a level 0 image, the image generated from 
the signals for the next highest luminosity value will be referred to as a level 1 image, etc. 

Figure 9 shows cartoon image 71 and photographic image 81, together with level 0 
images 72 and 82, level 1 images 73, and 83, level 2 images 74 and 84, and level 3 

20 images 75 and 85. It can be seen that the level 0 image 72 generated from the lowest 
luminosity signal for each block for the cartoon image 71 differs from the corresponding 
level 0 image 82 generated from a signal representing the photographic image 81 . The 
level 0 image 72 clearly comprises a plurality of outlines whereas the level 0 image 82 
does not. This is because even complex cartoons have outlines delineating the areas of 

25 one colour from the areas of another colour, even when the areas of colour are carefully 
shaded. Photographic images do not have such outlines. Small areas of the level 0 image 
82 may be mistakenly judged to contain outlines, however the majority of the image 82 
does not contain outlines. 

30 At step 61 of Figure 10 the signal representing a frame of video data is separated into 
block signals representing smaller areas of the frame. These need not be the same size 
areas as were produced at step 30 of Figure 6. Again, each area represented by a signal 
need not necessarily be the same size as each other area represented by a signal. At 
step 62 the number of dark pixels for each block is determined. Then at step 63 the 

35 number of blocks which are likely to contain outlines is determined by testing whether the 



percentage of dark pixels in a block less than a predetermined dark-threshold. At step 64 
a test is perfomied as to whether the number of blocks which are likely to contain outlines 
divided by the total number of blocks is greater than a predetermined outline-threshold. If 
the calculated ratio is greater than the outline-threshold then the signal is deemed to 

5 represent a cartoon, othenwise the signal is deemed not to represent a cartoon. In the 
embodiment described, the test for whether a block contains outlines is fairly simple. It 
would be possible to replace steps 62 and 63 by a more sophisticated algorithm which 
detects, for example, narrow bands of dark pixels, or an algorithm for detecting 
substantially parallel edges where pixels change firom dark to light, or vice versa, or to 

1 o implement a classifier using a neural network. 

As shown in Figure 5 an image classification program 109 according to the invention 
comprises a grey scale converter 130 which perfbmis steps 10 and 20 of Figure 6. an 
analysing means 140 which performs steps 30. 40 and 50 of Figure 6. and a likelihood 

1 5 generator 150 which performs step 60 of Figure 6. The analysing means 140 comprises a 
vector quantiser 142 which performs steps 30 and 40 of Figure 6 and a luminosity 
parameter generator 144 which perfomis step 50 of Figure 6. The likelihood generator 
150 comprises a low luminosity block signal generator 152. a block likelihood generator 
154 and a likelihood combiner 156. The vector quantiser 142 comprises a block signal 

20 generator 146 and a block signal vector quantiser 148. The luminosity parameter 
generator 144 comprises a low luminosity signal generator 132 and a percentage of low 
luminosity signals calculator 134. 

As will be understood by those skilled in the art. the image classification program 109 can . 
25 be contained on various transmission and/or storage mediums such as a floppy disc. CD- 
ROM, or magnetic tape so that the program can be loaded onto one or more general 
purpose computers or could be downloaded over a computer networi^ using a suitable 
transmission medium. 

30 Whilst the invention has been described with reference to a signal representing an image 
comprising a plurality of pixels, it will be appreciated that the detection of low luminosity 
outlines may equally well be performed on images for which the original source of the 
image does not represent the image as a plurality of pixels. 
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Unless the context clearly requires otherwise, throughout the description and the claims, 
the words "comprise", "comprising" and the like are to be construed in an inclusive as 
opposed to an exclusive or exhaustive sense; that is to say. in the sense of "including, but 
not limited to". 



CLAIMS 

1 . A method for classifying whether an image represents a cartoon, comprising the 
step of generating a likelihood in dependence on the presence of low luminosity outlines 
in the image. 

5 

2. A method according to claim 1 , and further comprising the steps of 
analysing the image to provide one or more parameters wherein one parameter 

relates to the luminosity of the image; and 

the generating step the generates the likelihood in dependence upon the value of 
10 said one parameter. 

3. A method according to claim 2, in which the Image comprises a plurality of pixels 
and the analysing step includes the sut)-step of vector quantising the image so that each 
pixel corresponds to one of a plurality of codes. 

15 

4. A method according to claim 3, in which the analysing step further comprises the 
sul>step of calculating the percentage of pixels conresponding to a one of the plurality of 
codes. 

20 5. A method according to claim 4, in which said one of the plurality of codes is a 
code which corresponds to pixels of low luminosity. 

6. A method according to any one of claims 3 to 5, in which the vector quantising 
sub-step comprises sub-steps of 

25 dividing the image into a plurality of blocks, each block comprising a subset of 

pixels in the image; and 
independently vector quantising each block. 

7. A method according to claim 6, In which the generating step comprises the sub- 
30 step of 

generating a block likelihood value for each of a plurality of blocks, the block 
likelihood representing the probability that the pixels in that 'block represent an image 
comprising one or more outlines. 
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8. A method according to claim 7. in which the generating step further comprises 
the sub step of combining a plurality of block likelihood values to provide the likelihood 
value for the image. 

9. An apparatus for classifying whether an image represents a cartoon, the apparatus 
comprising generating means for generating a likelihood in dependence on the presence 
of low luminosity outlines in the image. 

10. An apparatus according to claim 9. and further comprising 

means for analysing the image to provide one or more parameters wherein one 
parameter relates to the luminosity of the image; and 

the generating means receives in operation said one parameter and generates 
the likelihood in dependence upon the value of said one parameter. 

11. An apparatus according to claim 10, in which the analysing means comprises a 
vector quantiser which receives in operation a plurality of pixels comprising the image and 
outputs a plurality of codes each output code corresponding to each of the received 
pixels. 

12. An apparatus according to claim 11, in which the analysing means further 
comprises means for calculating the percentage of pixels corresponding to a one of the 
plurality of codes. 

13. An apparatus according to claim 12, in which said one of the plurality of codes is 
a code which conresponds to pixels of low luminosity. 

14. An apparatus according to any one of claims 11 to 13. in which the vector 
quantiser further comprises 

means for dividing the image into a plurality of blocks, each block comprising a 

subset of pixels in the image; and 

means for independently vector quantising each block. 

15. An apparatus according to claim 14, in which the generating means comprises 
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means for generating a block likelihood value for each of a plurality of blocks, the 
block likelihood representing the probability that the pixels in that block represent an 
image comprising one or more outlines. 

5 16. An apparatus according to daim 15. in which the generating means further 
comprises means for combining a plurality of block likelihood values to provide a 
likelihood value for the image. 

17. A data carrier loadable into a computer and carrying instmcllons for causing the 
1 0 computer to carry out the method according to any one of claims 1 to 8. 

18. A data carrier loadable into a computer and canying instructions for enabling the 
computer to provide the apparatus according to any one of claims 9 to 16. 

15 19. A method for classifying whether an image represents a cartoon substantially as 
described herein with reference to the accompanying drawings. 

20. An apparatus for classifying whether an Image represents a cartoon substantially 
as described herein with reference to the accompanying drawings. 
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ABSTRACT 
Cartoon Recognition 



A method of analysing an image for example from a video signal for the presence of 
outlines to detemiine whether the image is likely to represent a cartoon is based on the 
concept that cartoons are often characterised by the presence of dark outlines around 
objects in an improved version the signal is separated into signals representing smaller 
areas of the image before analysing each signal for the presence of dark outlines. 
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Figure 2a 



Figure 2b 
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Receive input signal 
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Convert signal to 
grey scale 
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Separate image into 
blocks 
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Independently vector 
quantise each block 



40 



Create iow luminosity' 
signal for the image 
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Use low luminosity 
signal to determine 
whether received signal 
represents a cartoon 
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Assign a code to 
each value 



For each code calculate 
the mean and standard 
deviation of values it 
represents 



Determine code with 
greatest standard 
deviation 



Assign a value of 
mean - 1/2 standard 
deviation to that code 
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Assign a value of 
mean +1/2 standard 
deviation to a new code 



Assign a code to 
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For each block in turn 



Set luminance value of each 
pixel to value associated with 
representative code 



For each code generate signal with 
luminance value of pixel to 
'white' if pixel not represented by 
that code 



All blocks done? 



For each block use the signal 
for the darkest code to generate a 
low luminosity signal for the image 
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count number of dark 
pixels for each block 
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